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ABSTRACT 

The use of Personal Response Systems (PRS) / Classroom Performance Systems (CPS) has 
expanded considerably since introduction in the early 2000s. Much of the exploration of the 
technology has focused on methodology, student participation, and student perception. This paper 
examines actual testing results over nine semesters to provide some insights to the impact of the 
technology on student grades. 
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INTRODUCTION 

n article appearing in the Chicago Tribune (Cohen, 2005) provided the motivation to begin implementation 
of the Personal Response System (PRS) “clickers” technology. The article focused primarily on the use of 
remote response units as an immediate means of student feedback; i.e., gaining insight as to whether 
students grasp the concepts being taught. 

Subsequent inquiry at Georgia Southern University’s College of Business Administration revealed a 
prototype program for the implementation of using clickers in the classroom. The PRS approach adopted at Georgia 
Southern University was the “Classroom Performance System” by elnstruction, hence, the abbreviation “CPS.” One 
of the participants in the program was Dr. Stephen Rutner, who had previous experience with clickers and had 
written an extensive article on the technology (Rutner, 2004). Rutner’s findings were that the overall impact on 
student learning was about one-half a letter grade improvement. He offered encouragement to take action with the 
technology and keep records of the results. 

After deciding that the use of clickers might be a viable way to improve student performance, an 
overarching strategic objective was determined: 

• Increase student understanding and retention of concepts. 

Use of the CPS approach supported several important intermediate objectives: 

Get feedback on student understanding of concepts in advance of actual testing. 

Create a strong impetus for students to read the cases before class. 

Increase the opportunity for student participation in the classroom. 

Increase the number of cases discussed in classroom without creating a workload monster for either the 
student or the instructor. 

PRIOR RESEARCH 

Most of the prior research on the use of remote access devices has focused on student satisfaction and 
student participation in the classroom. Hatch (2005) utilized clickers in an introductory anatomy and physiology 
course. Hatch found that 92% of the students felt that the clickers helped them see what they know or do not know 
and 83% felt that clickers were helping them to learn during class time. 
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Herreid (2005) found that clickers were ideally suited for case teaching using the Interrupted Case Method. 
The Interrupted Case method as used by Herreid integrates lecture material, case scenario material, student 
discussion with their neighbor, clicker questions, clarification of the answers, more lecture and data, then the cycle is 
repeated. In a general biology class with a class of 450 students, Herreid found that utilizing clickers after each 
stage of the case increased attendance, engaged students and improved class grades (but he doesn’t state by how 
much grades improved). 

Francisco (2006) found students in operations management courses were more favorable to the use of 
clickers than students in either accounting information systems courses or principles of retailing courses. Operation 
management students recommended the use of clickers at a higher rate and stated that clickers helped create a better 
learning environment. The variables that could have contributed to the deviation between classes could include 
instructor differences, different classroom methodology and differences in class personality. 

Carnaghan (2007) found strong student satisfaction with the technology. Carnaghan also found that 
students felt that the technology improved their performance. However, Carnaghan findings suggest that learning 
effects are limited when measured by exam performance. Positive exam effects were found only for exam questions 
that were similar to the in-class questions covered with the remote access devices. 

Stowell (2007) compared using clickers against both hand-raising in class and the use of response cards for 
students enrolled in an introductory psychology class. Stowell found that formal participation was greater with the 
use of clickers that with either response cards or hand-raising. However, Stowell found no significant differences on 
post class quiz performance between the three methods. Stowell did find that the actual quiz performance of the 
clicker group more closely reflected learning as their quiz scores were more aligned with in-class measurements of 
learning than either the hand-raising or response card measurements. 

In a 2008 paper, Cunningham (2008) reported mixed results. Utilizing clickers in two large-lecture 
sections of introductory accounting courses (about 840 students), he found that about 49% of students believed that 
the student response system should continue to be used while 28% felt that it should be discontinued. The remainder 
of the class either had no opinion or no response. Only 8% of students reported that they prepared more for class 
knowing the response system would be used to test his/her knowledge of the subject. Nearly 73% reported that they 
did not prepare more for class. Cunningham did find that over 55% of students found that the response system 
added interest to the class. Cunningham found that some of the negative student responses included such items as 
clickers do not work, the cost to the student, cannot participate when you forget your clicker and some people cheat 
by allowing friends to bring his/her clicker to class. 

Other research such as Barber (2007) compares the different types of clicker systems available and the 
strengths and weaknesses of each system. Barber does not recommend any system but does state institutions should 
adopt a single clicker system because it reduces costs for the student and reduces the number of systems the 
university needs to support. Standardization allows faculty to learn a single system and become more proficient 
with the technical capabilities of that system. 

CLASSROOM RESULTS 

Figure 1 shows that after the implementation of CPS, test scores improved by over one-half letter grade. 
Prior to utilizing CPS, the class average test grade over a three-semester period (Spring 05-Spring 06) was 71.83. 
After implementing CPS, the test average for the next three semesters (Fall 06-Fall 07) rose to 75.30. Over the 
following three semesters (Spring 08-Spring 09), the test grade average rose to 78.27. The overall increase of 6.44 
points represents better than a Vi grade increase - in this case from a low C to almost a B average. These findings 
aligned with Rutner’s previous findings that the use of the CPS improved test scores by about 14 letter grade. 

The data was accumulated from Introductory Operations Management classes taught by the same instructor 
over nine semesters. The average number of students per semester was 94, with a range of 40 to 142 (Figure 2). The 
number of classes each semester ranged from 1 to 4 with 3 and 4 being most typical. 
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It should be noted that CPS was not directly responsible for the results. It was the enabler. It allowed the 
introduction of a greater number of in-context business articles and cases and provided both feedback about 
students’ understanding and a measurable indication of their classroom participation. Most importantly, CPS 
provided an inducement for students to read over the material before it was discussed in class. The use of the CPS 
allowed the instructor to add more cases over time. Prior to utilizing CPS, the instructor was covering four cases per 
semester, after six semesters of using CPS, the number of cases covered had grown to 15. 
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Figure 1 


Test Grade Averages By Semester 



Figure 2: Number of Students and Cases Per Semester 


Semester 

# of Classes 

Total Students 

# of Cases 

Sp-05 

4 

121 

3 

Fa-05 

2 

67 

3 

Sp-06 

4 

96 

4 

Fa-06 

4 

128 

6 

Sp-07 

4 

142 

8 

Fa-07 

3 

72 

8 

Sp-08 

1 

40 

12 

Fa-08 

3 

101 

15 

Sp-09 

3 

82 

15 


Time investment - pre-classroom 

Preparing case questions for classroom use involved reading the case, and then one to two hours to put 
together suitable questions. Questions for cases were oriented toward three aspects: 

1. Have students read the case? 

2. Did students understand the concepts involved in the case? 

3. Is the student able to relate a concept to another situation? 
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In addition, each class must be set-up at the beginning of each semester. This involves logging on to the 
host site ( www.einstmction.com ) and creating each class by following a guided script. These steps are pretty 
routine and require Vi to 1 hour of time. One of the outputs is a Student Handout Sheet with the “class key” and 
instructions to register online to that specific class. A copy of this Student Handout was then provided to each 
student. The “class key” is different for each class and when the student goes on-line with elnstmction to register, it 
is the “class key” that links the student and the student’s “clicker” to the specific class. It can be expected that one 
student in about twenty-five will have a problem with registration - usually fairly easy to correct.. 

Using CPS in the Classroom 

CPS has far more facets and capabilities than were utilized. Pareto’s 80/20 rule was applied to take 
advantage of the 20% of the system that would provide 80% of the primary objectives. The 80/20 approach was 
intended to keep the instructor learning curve relatively short and minimize the time investment to get started. 

Using CPS in the classroom is straightforward. (Remember that only 20% of the CPS capabilities were 
being utilized.) Arriving at the classroom five minutes prior to the start of each class allowed adequate time to load 
the CPS Database from a flash-drive and initiate the software. Students merely had to turn on their response units, 
“clickers”, to be logged in for responses. 

Each question and multiple choice or true/false answers are projected on the screen (A typical case question 
used in the CPS can be found in Figure 3.) Students respond using their remote units. After each question is closed - 
i.e. student answers to that question are no longer accepted - the summary of overall responses to the question is 
shown. The response distribution provides immediate feedback and guides discussion. The CPS software 
automatically enables this process. 

The instructor had found that just assigning cases to be read did not get the cases read by the majority of 
students. Assigning written analysis meant reading and grading the student submissions, which in turn proved to be 
time consuming. Assigning more cases, and assuring they were read, required a more efficient approach. CPS 
offered an effective approach for giving students the impetus to read the cases/articles before class discussion and an 
easy way to track both the quality and quantity of participation. 

It was evident in the classroom that most students were reading the cases prior to class, resulting in better 
discussion and understanding. The CPS enabled the instructor to receive immediate feedback of the level of 
understanding and address any shortfalls immediately. This approach is far more interactive than finding out that 
students are not grasping a concept when they take a test several weeks later. Utilizing the CPS allowed the use of 
more cases to provide an example of a concept in action. The number of cases and articles used nearly quadrupled 
after the introduction of CPS. 

The CPS approach also makes it easier to connect concepts that are interrelated but presented at different 
times. For example, in operations management the relationships between types of strategy, appropriate process 
approach, and related issues about inventory and lean methodology are all normally covered at different times in the 
semester. By utilizing the CPS to ask case questions, the instructor can test the student’s understanding of the 
interrelationship of these topics. 


_ Figure 3: Typical CPS Question _ 

Students are assigned a short case to read. Then asked questions relating to the concept topic as well as the case This 
question happens to be about the use of Operations Management methodology in a barber shop (Heizer and Render, 
2006 ). 

Question: In the Barber Shop case, a service, the owner incorporated techniques and approaches that are conceptually 
similar to which production process type? 

A. Product Focus - high volume, low variety 

B. Process Focus - low volume, high variety 

C. Repetitive Focus - Use of modules (Darts & components previously prepared) 
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While anecdotal in nature, the instructor observed an increase in the quality of classroom discussion and the 
number of different students who would actively and intelligently participate in classroom interaction. 

CONCLUSION 

Remote response devices such as the CPS have the potential to improve student performance. The use of 
clickers in the classroom can serve as an enabler for students to improve their knowledge and retention of key 
concepts. Using CPS enabled test scores to increase by over Vi letter grade over a four year period. Utilizing CPS 
creates an impetus for students to read assigned cases before class and creates an environment that increases both the 
quantity and quality of classroom participation. 

The use of CPS allows more cases/material to be assigned which enhances learning. The instructor was 
able to nearly quadruple the amount of cases covered in a semester by utilizing CPS. The CPS gives both the 
instructor and students feedback on the comprehension of concepts prior to actual testing. Instructors are given 
immediate responses about student comprehension and can either move faster through the material or slow down 
and cover in more detail those subjects that students find more difficult. The use of CPS will continue to enable 
student learning as its usage becomes more widespread as technology continues to improve. 
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